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(57) Abstract 

A watermark is assumed to be embedded in a suspect image if there is a given amount of correlation between the suspect image (I) 
and the watermark (W). The reliability of such a watermark detection method is considerably improved by distorting (2) the suspect image 
in such a way that a distorted image (I*) is obtained which resembles the suspect image while the watermark is destroyed. The distorted 
image is then subtracted (3) from the suspect image to obtain a residual image (Q) in which the watermark is more dominant. The distortion 
may be a spatially-variant distortion such as stretching, shrinking, shearing or rotation. Embodiments are disclosed in which the distorted 
image is obtained by subjecting the suspect image to a non-linear transform or filter operation. 
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WO 00/26860 PCT/EP99/07926 
Watermark detection. 



FIELD OF THE INVENTION 

The invention relates to a method of detecting a watermark in a suspect image, 
comprising the steps of pre-processing the suspect image, determining the amount of 
correlation between the pre-processed image and the watermark, and generating a watermark 
5 detection signal in dependence upon said amount of correlation. The invention also relates to a 
device for performing the method. 

BACKGROUND OF THE INVENTION 

A method of detecting a watermark as defined in the opening paragraph is 

10 disclosed in International Patent Application WO-A 98/03014. As described in this 

Application, conventional watermark detection is based on determining the amount of 
correlation of a suspect image with the watermark to be detected. If the amount of correlation 
is less than a given threshold, the watermark is said to be absent, otherwise it is said to be 
present. The larger the correlation, the more reliable the detection and the more processing of 

15 the watermarked image is* allowed until the watermark cannot be detected anymore. 

To improve the reliability of the watermark detection, the suspect image is pre- 
processed. In the prior-art watermark detection method, the pre-processing includes predictive 
or matched filtering of the suspect image prior to the conventional watermark detection. Such 
a matched filter is a convolution filter, i.e. the filtered image is a spatially-invariant linear 

20 combination of global displacements of the same image. It is optimized to attenuate the 
frequencies for which the image dominates over the watermark. It works equally on the 
watermark and the underlying image. 

SUMMARY OF THE INVENTION 
25 It is an object of the invention to provide an improved method of detecting the 

watermark. 

To this end, the method is characterized in that said pre-processing comprises 
the steps of subjecting the suspect image to a non-convolution filter operation so as to obtain a 
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distorted but resembling image which has less correlation with the watermark, and subtracting 
said distorted image from the suspect image. 

A non-convolution filter is herein understood to mean a non-linear filter or a 
spatially-variant linear filter. 

The invention is based on the recognition that most watermarking schemes are 
not resistant to (even minor) non-linear or spatially variant distortions. Such distortions do not 
necessarily affect the perceptual appearance of the image, but destroy the watermark 
information. The distorted image resembles the original image but has less correlation with the 
watermark. This property is here exploited to improve the reliability of the watermark 
detection. By subtracting the distorted image from the suspect image, a difference image is 
obtained in which the ratio between the watermark information and the underlying residual 
image content is greatly enhanced. The performance of the subsequent conventional 
watermark detector is therefore considerably improved. 

Examples of non-linear filters are median filters, local minimum filters and 
local maximum filters. Examples of spatially variant operations are stretching, shrinking, 
shearing, rotation, motion compensation and region-based shifting. Various advantageous 
embodiments are defined in the appended claims and disclosed in the following description. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows a schematic diagram of a device for performing the method of 
detecting a watermark in accordance with the invention. 

Fig. 2 shows an embodiment of a device for detecting the watermark in 
accordance with the invention. 

Figs. 3A-3C show diagrams to illustrate the operation of the device which is 
shown in Fig. 2. 

Figs. 4 and 5 show further embodiments of devices for detecting the watermark 
in accordance with the invention. 

Figs. 6A-6C show diagrams to illustrate the operation of further embodiments 
of a distortion circuit shown in Fig. 1 . 

DESCRIPTION OF EMBODIMENTS 

Fig. 1 shows a device for detecting a watermark in a suspect image in 
accordance with the invention. The arrangement receives a suspect image I. Before applying 
the suspect image to a conventional watermark detector 1, the image is preprocessed by an 
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arrangement comprising a non-convolution filter 2 (hereinafter also referred to as distortion 
circuit) and a subtraction circuit 3 which subtracts the distorted image V from the suspect 
image I. The difference image Q is applied to the conventional watermark detector 1. 



5 circuit 12 and a threshold circuit 13. The multiplier 11 and the summation circuit 12 constitute 
a correlation circuit. The multiplier receives the difference image Q and an applied watermark 
W. The watermark W is a matrix of data values w;j that may have been added to the 
corresponding pixels Pijof an image P in a perceptually unobtrusive manner. 



10 suspect image I is watermarked (I=P+W) or not (I=P). To this end, the correlation circuit 

(11,12) calculates the inner product <Q,W> of the difference image Q and the watermark W, 
defined as: 



in which Nj and N 2 are the width and height of the image in number of pixels, qij denotes the 
15 pixel value of the difference image Q at position (i j), and wy is the watermark data value at 

pixel position (i,j). It will be appreciated that, if the image signal is an analog signal, the 

summation circuit 12 will be an integrator. 

The calculation of the correlation <Q,W> is often carried out in the Fourier 

domain. Because the dynamic range of the difference image Q in the Fourier domain is small, 
20 the difference image can be represented by integers rather than floating point numbers. This 

reduces the hardware complexity considerably. 

The output <Q,W> of the summation circuit 12 represents an amount of 

correlation between the input signal Q and the applied watermark W. The threshold circuit 13 

determines whether said amount of correlation exceeds a predetermined threshold. If that is 
25 the case, detection signal D=l is generated to indicate the presence of the watermark. 

Otherwise the watermark is said to be absent (D=0). 

Fig. 2 shows an embodiment of the watermark detection device in accordance 

with the invention. In this embodiment, the suspect image I is one of a sequence of images 

together constituting a motion video signal. The distortion circuit 2 comprises a motion 
30 estimation circuit 21, a memory 22. and a motion compensation circuit 23. An earlier image of 

the sequence is available at the output of the memory 22. The earlier image (hereinafter 

referred to as reference image) may be the previous image or spaced a number of images apart. 

It may have the same watermark as the current image. The reference image and the current 



The conventional watermark detector 1 comprises a multiplier 11, a summation 



The task of the device, which is shown in Fig. 1, is to detect whether the 
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image are applied to the motion estimation circuit 21. Motion estimation is a well-known 
operation in the field of image coding and needs no detailed explanation. It suffices to mention 
that the motion estimation circuit 21 searches, for each image block of the current image I, the 
most resembling image block in memory 22, and generates a motion vector mv indicating the 

5 relative position of the block thus found. It is important to note that the motion estimation is 
hardly affected by the embedded watermark, if any, because the image content is dominant. 

The motion vectors mv and the reference image are then applied to the motion 
estimation circuit 23 to obtain a prediction for the current image. The prediction image V 
resembles the current image but its watermark has been shaken up by the motion vectors. This 

10 is shown by way of example in Figs. 3A-3C. In Fig. 3 A, the motion vectors generated by the 
motion estimation circuit 21 are shown as arrows. For example, motion vector 31 denotes that 
image block 33 of the reference image has been found to closely resemble image block 32 of 
current image I. Fig. 3B shows the watermark, which is the same in the current image and the 
reference image. For reasons of clarity, the watermark pattern is extremely simple in this 

15 example. Image content is not shown in the Figure although it dominates over the watermark 
in practice. Fig. 3C shows the prediction image V generated by the motion compensation 
circuit. Image block 34 of this prediction image is the same as image block 33 of the reference 
image (see Fig. 3B) and thus resembles block 32 of the current image, but the watermark has 
been shifted. Again, the dominant image content is not shown in the Figure. 

20 The difference image Q, obtained after subtracting the prediction image V from 

the suspect image I can be written as: 

Q=i-r 

= (P + W)-(F+W) 
= (P-P') + W-W 

in which P 1 and W denote the motion-compensated image content and the motion- 
compensated watermark (cf. Fig. 3C), respectively. 

25 In case of perfect motion estimation, the image residue (P-P') is very small, so 

that the correlation calculated by the watermark detector 1 can be approached by: 

<Q,W > = <1V,W >-<W % W % > 
As can easily be understood with reference to Figs. 3B and 3C, the term <W,W> which 
represents the correlation between the watermark W and its motion-compensated counterpart 

30 W is low, provided that the motion vectors are sufficiently different. The correlation <Q,W> 
is thus mainly defined by the desired term <W,W>. 
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It should be noted that motion estimation (21), motion compensation (23), and 
subtracting (3) have often already been carried out at the encoding end of a video transmission 
chain. For example, MPEG encoded video signals include predictively encoded (P and B-) 
pictures. They are transmitted as residual images plus motion vectors. In view thereof, it will 
be understood that a watermark in a received MPEG video signal can best be detected by 
subjecting the residual image to watermark detection. 

The embodiment of the watermark detection device shown in Fig. 2 suffers 
from a reduced performance if there is no motion between the suspect image and the reference 
image (all motion vectors are zero). The performance is also reduced if large image areas are 
subject to the same motion (all motion vectors are the same). Fig. 4 shows an embodiment of 
the device in which this problem is alleviated. The embodiment differs from the embodiment 
shown in Fig. 2 in that a circuit 24 for randomly modifying the motion vectors mv is inserted 
between the motion estimator 21 and the motion compensator 24. It is thereby achieved that 
the watermark W and its motion-compensated counterpart W* are decorrelated irrespective of 
whether (large parts of) the image is subject to uniform motion or not. A disadvantage of 
randomizing the motion vectors is that the residual image content (P-P' in the formulas above) 
increases, even if the motion estimation is perfect. This can be solved, if necessary, by 
modifying the motion vectors only in image areas in which they are substantially zero or 
identical. 

It has been found that the performance of the watermark detection is not 
dramatically affected by the motion vector randomization. This observation has led to a further 
embodiment which is shown in Fig. 5. In this embodiment, the motion estimation circuit has 
been omitted and replaced by a circuit 25 for randomly generating motion vectors. Not only is 
this embodiment cost-effective in view of the absence of an expensive motion estimator, it can 
also be used to detect a watermark in isolated images without any reference to earlier images 
having the same watermark. To this end, the motion compensation circuit 23 receives the 
suspect image instead of an earlier reference image. In fact, the distortion circuit 2 in Fig. 5 
generates a distorted image V by randomly displacing regions of the suspect image I as shown 
in Fig. 3C. 

Figs. 6A-6C show diagrams to illustrate further examples of spatially-variant 
linear filter operations carried out by the distortion circuit 2. In the Figs., the rectangle ABCD 
represents the suspect image I and the polygon A'B'C'D' represents the distorted image l\ In 
Fig. 6A. the distorted image is obtained by stretching (or shrinking if the image size is 
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decreased). In Fig. 6B, the distorted image is obtained by shearing. In Fig. 6C, the distorted 
image is obtained by rotation. 

In a further embodiment of the device in accordance with the invention, the 
distortion filter 2 is a non-linear filter. An example of such a filter is a median filter. A median 

5 filter is a filter which receives a series of N input samples, re-arranges the series in order of 
decreasing or increasing values, and selects the middle sample of the re-arranged series. Here, 
a median filter is also understood to mean a filter which selects the n th (not necessarily the 
middle) sample of the series. It has been found that median filtering of a watermarked image 
yields a closely resembling image as far as image content is concerned, but largely destroys 

10 the embedded watermark. Thus, after subtracting the filtered image V from the suspect image 
I, the image information is reduced and the watermark is retained. In the embodiment shown 
in Fig.6, the median filter is a 3*3 median filter. Each pixel of the suspect image is replaced by 
the median of the series comprising the original pixel and its eight neighboring pixels. 

In summary, a watermark is assumed to be embedded in a suspect image if 

15 there is a given amount of correlation between the suspect image (I) and the watermark (W). 
The reliability of such a watermark detection method is considerably improved by distorting 
(2) the suspect image in such a way that a distorted image (F) is obtained which resembles the 
suspect image while the watermark is destroyed. The distorted image is then subtracted (3) 
from the suspect image to obtain a residual image (Q) in which the watermark is more 

20 dominant. The distortion may be a spatially-variant linear distortion such as stretching, 
shrinking, shearing or rotation, or a non-linear transform or filter operation. 
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1. A method of detecting a watermark in a suspect image, comprising the steps of 
pre-processing (2,3) the suspect image, determining (11,12) the amount of correlation between 
the pre-processed image and the watermark, and generating (13) a watermark detection signal 
in dependence upon said amount of correlation, characterized in that said pre-processing (2,3) 

5 comprises the steps of subjecting the suspect image (I) to a non-convolution filter operation 
(2) so as to obtain a distorted but resembling image (V) which has less correlation with the 
watermark, and subtracting (3) said distorted image from the suspect image. 

2. A method as claimed in claim 1, wherein the suspect image is one of a sequence 
10 of images, the method comprising the steps of estimating motion (21) between the suspect 

image and another image (22) of said sequence to obtain motion vectors, the distorted image 
being obtained by applying motion compensation (23) to said other image using said motion 
vectors. 

15 3. A method as claimed in claim 2, further comprising the step of randomly 

modifying (24) said motion vectors at least in image areas in which the motion vectors are 
zero or identical. 

4. A method as claimed in claim 1, wherein the suspect image is divided into a 
20 plurality of image regions (33), the distorted image being obtained by differently shifting (31) 

said regions. 

5. A method as claimed in claim 1, in which the suspect image is a block-based 
predictively encoded image of a motion video signal comprising a residual 

25 image and motion vectors, characterized in that said residual image constitutes the pre- 
processed image. 

6. A method as claimed in claim 1, wherein said step of deriving the distorted 
image comprises subjecting the suspect image to a non-linear filter operation. 
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7 a method as claimed in claim 6, wherein said non-linear filter is a median filter. 

g a device for detecting a watermark in a suspect image, comprising means (2,3) 

for pre-processing the suspect image, means (11,12) for determining the amount of correlation 
between the pre-processed image and the watermark, and means (13) for generating a 
watermark detection signal in dependence upon said amount of correlation, characterized in 
that said means (2) for pre-processing comprises means (2) arranged to derive from the 
suspect image a distorted but resembling image which has less correlation with the watermark, 
and means (3) for subtracting said distorted image from the suspect image. 
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